The present invention relates generally to quality control (QC) compliance monitoring and more specifically to a system (analytical testing process) for automated exception-based quality control compliance for point-of-care devices.
All clinical laboratories in the United States must comply with the Clinical Laboratory Improvement Amendments of 1988 (CLIA ""88). CLIA ""88 has established the minimum standards for all laboratory testing, including specific regulations for quality control. Although CLIA ""88 does nor explicitly recommend a method for determining when a system is out of control, the federal law does state that laboratories must:
perform control procedures using at least two levels of control materials each day of testing
establish written procedures for monitoring and evaluating analytical testing processes
follow the manufacturer""s instructions for quality control
To achieve the goal of quality control, including the precision and accuracy of test results, it is necessary to be able to detect errors within the system as soon as possible. Before designing an error detecting system, something must be known about the nature of the errors to be detected. Typically, the errors in a quantitative system are random errors and systematic errors.
Random errors are always present to a measurable degree in any system given a set of circumstances-glucose meters (the devices), operators, test strips (the reagent), and control solutions (the control material), for example. The amount of random error, sometimes referred to as precision, is usually measured by the standard deviation (SD) and the coefficient of variation (CV). The SD measures the scatter (for variability around the true value) in the data points (test results), while the CV is the standard deviation expressed as a percent.
The other type of error is systematic error. These errors, of which shifts and trends are included, occur in one direction away from the true value and are measured by using the mean. Accuracy is the term used when referring to how close a test result is to the true value.
Not only should the error detecting system be able to detect these two types of errors, but it should be able to tell us whether the error is random or systematic because this leads the analysis in a direction which is highly significant.
The manufacturer""s stated QC ranges give an indication of where the mean and QC limits may be, but the manufacturer data is not considered an appropriate substitute for a mean and QC limits determined from the institution""s own established data. Each institution should determine the performance of their measurement system and set an appropriate mean and QC limits for the controls based on their own data. New reagent and/or control material should be analyzed for each analyte in parallel with the reagent and/or control material currently in use.
The National Committee for Clinical Laboratory Standards (NCCLS) recommends that as a minimum, 20 data points from 20 or more separate runs be obtained to determine an estimate of mean and standard deviation for each level of control material. A run is typically defined in terms of a length of time or a number of samples analyzed. Better estimates of both mean and standard deviation can be achieved when more data is collected. Additionally, the more controls run, the easier it is to detect true changes in the measurement system.
It is important to include all valid data points attained with the selected collection method. For example, if values outside of 2 SD are not included in the data, an artificially small estimate of variability may be calculated.
If data collection is to be representative of future system performance, sources of variation that are expected and determined acceptable may be included during the data collection period. These may include multiple devices, reagent lots, multiple control material lots, and multiple operators to name a few examples.
The Gaussian distribution, or bell-shaped curve, is the most frequently used model when analyzing clinical data. Using the true standard deviation, statistical theory shows that 99.73% of the data will fall within xc2x13 SD of the mean, 95.44% will fall within xc2x12 SD of the mean, and 68.26% will fall within xc2x11 SD of the mean for each level of control material. (Standard deviation estimates from actual data may vary from the true standard deviation.)
After determining the mean and SD of a measuring system, many institutions will decide to set control limits as some multiple of the SD around the mean, for example, at xc2x12 SD or xc2x13 SD to determine when a system is out of control. The problem with this single-rule method is that even if there is no change in the performance of the system and everything is operating as expected (system is in control), you will still have 4.56% (100xe2x88x9295.44=4.56) of values fall outside the xc2x12 SD limits. These points are considered false rejections, and the more data points you collect, the higher the number of false rejections encountered. Thus, while the xc2x12 SD offers a very sensitive method to detecting a change, is also presents a real problem-a high rate of false rejection.
Multirule quality control methods use a combination of control rules to more accurately decide whether analytical runs are in control or out-of-control. Unlike the 2-SD or 3-SD limit rules described above, the Westgard Multirule Procedure (Westgard 1938) uses six different control rules to judge the acceptability of an analytical run. The advantages of a multirule QC method are that false rejections can be kept low while at the same time maintaining high error detection.
The following summarizes the individual Westgard 1938 control rules:
To perform multirule QC, start by collecting control data and establish the means and SDs for each level of control material. If performing QC manually (plotting and interpreting data without the use of a computer program), create a Levey-Jennings chart and draw lines at the mean, xc2x11 SD, xc2x12 SD, and xc2x13 SD.
In manual applications, the 12s, rule should be used as a warning to trigger application of the other rules. It indicates that one should look carefully before proceeding. Stop if the 13s, rule is broken. Stop if the 22s, rule is broken. Stop if the R4s rule is broken. Often the 41s, and 10x rules must be used across runs in order to get a sufficient number of control measurements needed to apply the rules.
A software program should be able to select the individual rejection rules on a test-by-test basis to optimize the performance of the QC procedure on the basis of the precision and accuracy observed for each analytical method and the quality required by the test.
A computer-implemented method to process point-of-care (POC) information for potential QC compliance issues. A system and method for implementation of traditional laboratory analyzer based QC compliance in POC environments is disclosed. A specific system and method to analyze data from POC testing to identify when the testing exceeds the variation expected under stable operation (i.e., the testing is xe2x80x9cout of controlxe2x80x9d) is disclosed. This system and method is characterized by solving the QC compliance problem for POC devices by individuals not trained in traditional laboratory practices. This also provides the capability in real-time or near real-time to analyze POC testing information regarding the performance of each POC device, reagent kit (i.e., one kit per analyte tested) and/or lot, and operator so one can respond quickly to a particular device, reagent kit and/or lot, or operator that is not performing properly.
The disclosed invention builds upon the QC control testing and warning/lockout functionality currently performed and supported by many POC devices. Typically, QC enforcement is specified by a POC administrator (a.k.a. coordinator) so that required QC test(s) must be performed at designated intervals to avoid a QC warning or lockout. These POC devices provide a method to warn/flag or lockout patient testing if any required QC test fails (i.e., not within expected QC limits) or is not run as scheduled. Additionally, the disclosed method builds upon connectivity solutions which have become industry standard for POC devices. These solutions allow for the transfer of QC and patient test results from POC devices, the transfer of configuration information to POC devices, and the transfer of Patient (and QC) test results to laboratory and/or hospital information systems.
In summary, the present system and method provide traditional laboratory analysis based QC compliance processing in POC environments. The unique characteristics of a POC environment include multiple and possible widely distributed testing devices (typically with limited processing capability); utilizing multiple reagent kits and/or lots; and, use by multiple operators to perform various analytical testing at or near the site of patent care.
The invention provides automated QC compliance checking to verify POC testing is performing properly (i.e., a method to monitor and evaluate the QC test results of a POC environment and to identify and alert the POC administrator(s) when the testing exceeds the variation expected under stable operation). Additionally, to automate the identification of the specific cause of the variation (e.g., device(s), reagent kits and/or lot(s), and/or operator(s)); the current industry standard is a manual review of QC compliance and outlier reports and manual intervention when system is found to be xe2x80x9cout of controlxe2x80x9d.
The invention provides exception-based QC compliance for POC environments to ensure timely notification and response to xe2x80x9cout of controlxe2x80x9d POC testing, to improve the quality of patent testing performed at or near the site of patient care. Additionally, to reduce the need and the time to manage POC information (i.e., timely access to accurate and complete QC information is critical for cost effective quality health care delivery).
The invention defines rules for detecting random and systemic errors within QC test results (i.e., QC compliance settings which are similar to traditional Westgard 1938 rules) and to automatically log the QC compliance settings and/or changes.
The invention detects potential inaccurate patient results and optionally holds for review any POC collected patient results when associated with QC compliance exceptions (i.e., optionally delay the reporting and/or transfer to an information system of potential inaccurate patient results).
The invention logs and maintains QC compliance issues (i.e., exceptions to defined rules).
The invention alerts POC administrator (a.k.a. coordinator) to potential QC compliance issues (possible options include status display, electronic mail, faxing, and paging). (See data flow #7).
The invention clears QC compliance alerts when reviewed by POC administrator (a.k.a. coordinator).
The invention defines rules for detecting and optionally disabling the cause of the variation (e.g., number of QC compliance issues within a specified time frame for devices, reagent kits and/or lots, and/or operators) and to automatically log the QC compliance settings and/or changes.
The invention automatically disables questionable POC devices (i.e., a feedback loop built upon connectivity solutions and device configuration). Additionally, the ability to automatically remove from use (via device configuration) any xe2x80x9cout of controlxe2x80x9d reagent kits and/or lots, and/or operators.
The invention releases for or stops from reporting and/or transfer to an information system patent results associated with QC compliance exceptions (following the review of associated QC compliance exception(s)).
The invention provides automated access (e.g., browse) to current and reviewed QC compliance exceptions with detailed QC compliance reporting (e.g., Levey-Jennings chart and associated statistical analysis) to support accreditation requirements and continual improvement in the quality of care provided by POC testing. (See data flow #7, #8, #9, and #12).
The invention supports automated accreditation validation (e.g., CAP or JACHO) via logging of QC compliance settings, QC compliance exceptions, and the review of QC compliance exceptions, and storing the above for review by the administrator and/or an auditor.
Point-of-care, or near the site of patient care, testing is a technological innovation that has demonstrated the potential for improving patient care (i.e., improves the quality and outcomes of care while decreasing cost and length of stay for patients), assuming accurate and precise patient results are obtained in near real-time. The ultimate goal of quality control (QC) compliance is to ensure accurate and precise patient results whether generated by point-of-care (POC) devices or performed by traditional laboratory analyzers. However, to ensure POC testing generates accurate and precise patient results, an automated method to monitor and evaluate the QC test results of a POC environment, and to identify and respond when the testing exceeds the variation expected under stable operation is required. Additionally, to ensure timely notification and response to xe2x80x9cout of controlxe2x80x9d POC testing, an exception-based method is required.
Other features and advantages of the inventive system will be apparent to a person of skill in this art who studies the following description of an exemplary embodiment.